The present invention relates to graphical user interfaces and more particularly to graphical user interfaces for pen-based computing.
Pen-based computing is becoming increasingly popular. The user of a pen-based computer inputs commands and data into the computer using a pen or stylus, typically by printing or writing characters with the pen or stylus on the computer's display screen in a manner analogous to printing or writing with a pen on ordinary paper. Certain commands can be entered as pen strokes (gestures); for example, a word may be deleted by crossing it out, or a file may be opened by writing an "o" on its representative icon. Examples of pen-based computing are found in systems ranging from palm-sized personal digital assistants, through slate-sized machines such as notepad computers, and up to wall-sized interactive displays. Also, pen-based interfaces can sometimes be added or retrofitted to computer systems not originally designed to be pen-based, such as certain personal computer systems.
The usefulness of pen-based computing is limited because of the limited capabilities of character recognition software. In many pen-based systems, the user cannot write in cursive handwriting, but must print. This slows down input. Furthermore, the system has difficulty in recognizing printed characters. Consequently, the user must print carefully (viz., slowly) and often must print in special windows and within designated boxes, and the system can require special training for each individual user. Even so, character recognition performance can be relatively poor. In addition, pen-based systems that use character recognition do not adapt well to run on conventional computer workstations equipped with a mouse, for example.
Because of the limitations of the character-recognition approach to pen-based input, an alternative approach is sometimes used. In this approach, a graphical keyboard, visually analogous to the conventional mechanical (typewriter) keyboard commonly used for input in non-pen-based computer systems, appears on the computer display. The user enters text by using the stylus to tap on desired keys of the graphical keyboard.
The graphical-keyboard approach offers certain advantages over the character-recognition approach. Entering data through a graphical keyboard can be faster than printing (especially with numeric keypads), can be less susceptible to errors when properly implemented (i.e., when the graphical keys are not too small), and does not require the system to be specially trained for each individual user. Moreover, the approach is simple to implement and integrate into existing software applications.
Unfortunately, known graphical keyboards suffer from numerous drawbacks. Although the graphical keyboard is visually analogous to a mechanical keyboard, it is functionally far more limited. With a graphical keyboard, a user cannot "touch type" using all fingers of both hands, but instead must "hunt and peck" with the pen or stylus, as if typing with one finger on a mechanical keyboard. Typing speed on the graphical keyboard is constrained by the time required for the user to move the finger/stylus from key to key on the keyboard. Also, the user cannot tap multiple keys simultaneously. In particular, the user cannot tap a modifier key, such as the "shift," "ctrl," "option," or "alt" key, at the same time as an ordinary key, such as a letter or number key, but instead must first tap the modifier key and then tap the letter or number key. In some systems the user must tap the modifier key once to set the modifier, tap the letter or number key, and then tap the modifier key again to release the modifier (for example, tap "shift," "a," "shift" to input an uppercase "A". This can be especially awkward when multiple modifiers must be composed to form a single character (for example, a doubly modified character such as ctrl-alt-x or a triply modified character such as ctrl-alt-X).
Another problem with the graphical keyboard approach is that it places a heavy visual load on the user. Whereas with a mechanical keyboard, the user can memorize commonly used sequences of keystrokes and perform them "by feel" without looking at the keyboard, with the graphical keyboard the user typically must look at the display before each tap of the stylus to ensure that the stylus is correctly placed. Furthermore, the graphical keyboard tends to take up a large portion of the screen display, in part because the keyboard contains many or all of the modifier and function keys found in its mechanical counterpart.
Because of its drawbacks, the graphical-keyboard approach to pen-based input is often considered cumbersome, and is used in pen-based computing systems only as a fallback when character recognition fails or is insufficiently reliable. The character-recognition and graphical-keyboard approaches to pen-based computing are typically seen as mutually exclusive alternatives, not as complements. In particular, it has not been possible to combine the intuitive ease of use promised (if not always afforded) by the character-recognition approach with the speed, robustness and ease of implementation offered by the graphical keyboard approach.